Method and apparatus for enhancing images via white pop-out

ABSTRACT

A user equipment (UE) includes a receiver and a processor. The receiver is configured to receive a standard dynamic range (SDR) image and metadata related to an HDR image. The processor is configured to identify relevant portions of the SDR image to be enhanced based on the metadata related to the HDR image. The processor is also configured to increase an intensity of the relevant portions of the SDR image to create an enhanced SDR image. The processor is also configured to output the enhanced SDR image to a display.

CROSS-REFERENCE TO RELATED APPLICATION(S) AND CLAIM OF PRIORITY

This application claims priority under 35 U.S.C. §119(e) to U.S.Provisional Patent Application No. 62/222,054 filed on Sep. 22, 2015.The above-identified provisional patent application is herebyincorporated by reference in its entirety.

TECHNICAL FIELD

This disclosure relates generally to image enhancement. Morespecifically, this disclosure relates to a method and apparatus forenhancing images via white pop-out.

BACKGROUND

Currently, new High Dynamic range (HDR) cameras and displays arebecoming prominent. HDR cameras can capture low to high intensities suchas from 0.01 to around 10,000 nits. While studios are primarily creatingHDR content, due to the high cost of the HDR cameras and displays, theyhave not yet reached normal consumers

Furthermore, a majority of the content is Standard Dynamic Range (SDR).While there are many algorithms for inverse tone mapping to convert theSDR content to HDR for displaying on HDR displays, most consumers stillhave SDR displays. It is also expected that the HDR displays will beconsiderably expensive than the SDR displays for a long time, and thenormal consumer will have access to SDR displays only.

SUMMARY

This disclosure provides method and apparatus for image enhancement viawhite pop-out.

In a first embodiment, a user equipment (UE) includes a receiver and aprocessor. The receiver is configured to receive a standard dynamicrange (SDR) image and metadata related to an HDR image. The processor isconfigured to identify relevant portions of the SDR image to be enhancedbased on the metadata related to the HDR image. The processor is alsoconfigured to increase an intensity of the relevant portions of the SDRimage to create an enhanced SDR image. The processor is also configuredto output the enhanced SDR image to a display.

In a second embodiment, a method provides image enhancement. The methodincludes receiving a standard dynamic range (SDR) image and metadatarelated to an HDR image. The method also includes identifying relevantportions of the SDR image to be enhanced based on the metadata relatedto the HDR image. The method also includes increasing an intensity ofthe relevant portions of the SDR image to create an enhanced SDR image.The method also includes outputting the enhanced SDR image to a display.

Other technical features may be readily apparent to one skilled in theart from the following figures, descriptions, and claims.

Before undertaking the DETAILED DESCRIPTION below, it may beadvantageous to set forth definitions of certain words and phrases usedthroughout this patent document. The term “couple” and its derivativesrefer to any direct or indirect communication between two or moreelements, whether or not those elements are in physical contact with oneanother. The terms “transmit,” “receive,” and “communicate,” as well asderivatives thereof, encompass both direct and indirect communication.The terms “include” and “comprise,” as well as derivatives thereof, meaninclusion without limitation. The term “or” is inclusive, meaningand/or. The phrase “associated with,” as well as derivatives thereof,means to include, be included within, interconnect with, contain, becontained within, connect to or with, couple to or with, be communicablewith, cooperate with, interleave, juxtapose, be proximate to, be boundto or with, have, have a property of, have a relationship to or with, orthe like. The term “controller” means any device, system or part thereofthat controls at least one operation. Such a controller may beimplemented in hardware or a combination of hardware and software and/orfirmware. The functionality associated with any particular controllermay be centralized or distributed, whether locally or remotely. Thephrase “at least one of,” when used with a list of items, means thatdifferent combinations of one or more of the listed items may be used,and only one item in the list may be needed. For example, “at least oneof: A, B, and C” includes any of the following combinations: A, B, C, Aand B, A and C, B and C, and A and B and C.

Moreover, various functions described below can be implemented orsupported by one or more computer programs, each of which is fixatedfrom computer readable program code and embodied in a computer readablemedium. The terms “application” and “program” refer to one or morecomputer programs, software components, sets of instructions,procedures, functions, objects, classes, instances, related data, or aportion thereof adapted for implementation in a suitable computerreadable program code. The phrase “computer readable program code”includes any type of computer code, including source code, object code,and executable code. The phrase “computer readable medium” includes anytype of medium capable of being accessed by a computer, such as readonly memory (ROM), random access memory (RAM), a hard disk drive, acompact disc (CD), a digital video disc (DVD), or any other type ofmemory. A “non-transitory” computer readable medium excludes wired,wireless, optical, or other communication links that transporttransitory electrical or other signals. A non-transitory computerreadable medium includes media where data can be permanently stored andmedia where data can be stored and later overwritten, such as arewritable optical disc or an erasable memory device.

Definitions for other certain words and phrases are provided throughoutthis patent document. Those of ordinary skill in the art shouldunderstand that in many if not most instances, such definitions apply toprior as well as future uses of such defined words and phrases.

BRIEF DESCRIPTION OF THE DRAWINGS

For a more complete understanding of this disclosure and its advantages,reference is now made to the following description, taken in conjunctionwith the accompanying drawings, in which:

FIG. 1 illustrates an example wireless network according to thisdisclosure;

FIG. 2 illustrates an example eNodeB (eNB) according to this disclosure;

FIG. 3 illustrates an example user equipment (UE) according to thisdisclosure;

FIG. 4 illustrates an example system for processing an image accordingto an embodiment of this disclosure;

FIG. 5 illustrates an example image to be enhanced according to anembodiment of this disclosure;

FIG. 6 illustrates an example mask for enhancement according to anembodiment of this disclosure;

FIGS. 7A and 7B illustrate example charts for enhancement portionsaccording embodiments of this disclosure;

FIG. 8 illustrates an example process for post-processing enhancement ofan image according to an embodiment of this disclosure;

FIG. 9 illustrates an example chart for input and output luminancecharacteristics according embodiments of this disclosure; and

FIG. 10 illustrates an example system using an HDR image for processingan SDR image according to an embodiment of this disclosure.

DETAILED DESCRIPTION

FIGS. 1 through 10, discussed below, and the various embodiments used todescribe the principles of this disclosure in this patent document areby way of illustration only and should not be construed in any way tolimit the scope of the disclosure. Those skilled in the art willunderstand that the principles of this disclosure may be implemented inany suitably arranged wireless communication system.

FIG. 1 illustrates an example wireless network 100 according to thisdisclosure. The embodiment of the wireless network 100 shown in FIG. 1is for illustration only. Other embodiments of the wireless network 100could be used without departing from the scope of this disclosure.

As shown in FIG. 1, the wireless network 100 includes an eNodeB (eNB)101, an eNB 102, and an eNB 103. The eNB 101 communicates with the eNB102 and the eNB 103. The eNB 101 also communicates with at least oneInternet Protocol (IP) network 130, such as the Internet, a proprietaryIP network, or other data network.

The eNB 102 provides wireless broadband access to the network 130 for afirst plurality of user equipments (UEs) within a coverage area 120 ofthe eNB 102. The first plurality of UEs includes a UE 111, which may belocated in a small business (SB); a UE 112, which may be located in anenterprise (E); a UE 113, which may be located in a WiFi hotspot (HS); aUE 114, which may be located in a first residence (R); a UE 115, whichmay be located in a second residence (R); and a UE 116, which may be amobile device (M) like a cell phone, a wireless laptop, a wireless PDA,or the like. The eNB 103 provides wireless broadband access to thenetwork 130 for a second plurality of UEs within a coverage area 125 ofthe eNB 103. The second plurality of UEs includes the UE 115 and the UE116. In some embodiments, one or more of the eNBs 101-103 maycommunicate with each other and with the UEs 111-116 using 5G, LTE,LTE-A, WiMAX, WiFi, or other wireless communication techniques.

Depending on the network type, other well-known terms may be usedinstead of “eNodeB” or “eNB,” such as “base station” or “access point.”For the sake of convenience, the terms “eNodeB” and “eNB” are used inthis patent document to refer to network infrastructure components thatprovide wireless access to remote terminals. Also, depending on thenetwork type, other well-known terms may be used instead of “userequipment” or “UE,” such as “mobile station,” “subscriber station,”“remote terminal,” “wireless terminal,” “television” or “user device.”For the sake of convenience, the terms “user equipment” and “UE” areused in this patent document to refer to a television, display, monitor,or other such wired or wireless devices. The UE can be in communicationwith another UE, such as a mobile device, or other television.

Dotted lines show the approximate extents of the coverage areas 120 and125, which are shown as approximately circular for the purposes ofillustration and explanation only. It should be clearly understood thatthe coverage areas associated with eNBs, such as the coverage areas 120and 125, may have other shapes, including irregular shapes, dependingupon the configuration of the eNBs and variations in the radioenvironment associated with natural and man-made obstructions.

One or more embodiments of this disclosure provide methods andapparatuses to performing post-processing of SDR content to give an HDReffect on SDR displays. In one example, the white portions of the SDRimage are enhanced to have a brighter (or closer to pure white) nature.As used herein, one or more embodiments of this disclosure refers to anSDR or HDR image. However, different embodiments of this disclosure canalso be used with video. When referencing an image herein, whether SDRor HDR, the different embodiments of this disclosure could be referringto a frame within a video for a given frame rate (number of pictures perunit of time).

Although FIG. 1 illustrates one example of a wireless network 100,various changes may be made to FIG. 1. For example, the wireless network100 could include any number of eNBs and any number of UEs in anysuitable arrangement. Also, the eNB 101 could communicate directly withany number of UEs and provide those UEs with wireless broadband accessto the network 130. Similarly, each eNB 102-103 could communicatedirectly with the network 130 and provide UEs with direct wirelessbroadband access to the network 130. Further, the eNB 101, 102, and/or103 could provide access to other or additional external networks, suchas external telephone networks or other types of data networks.

FIG. 2 illustrates an example eNB 102 according to this disclosure. Theembodiment of the eNB 102 illustrated in FIG. 2 is for illustrationonly, and the eNBs 101 and 103 of FIG. 1 could have the same or similarconfiguration. However, eNBs come in a wide variety of configurations,and FIG. 2 does not limit the scope of this disclosure to any particularimplementation of an eNB.

As shown in FIG. 2, the eNB 102 includes multiple antennas 205 a-205 n,multiple RF transceivers 210 a-210 n, transmit (TX) processing circuitry215, and receive (RX) processing circuitry 220. The eNB 102 alsoincludes a controller/processor 225, a memory 230, and a backhaul ornetwork interface 235.

The RF transceivers 210 a-210 n receive, from the antennas 205 a-205 n,incoming RF signals, such as signals transmitted by UEs in the network100. The RF transceivers 210 a-210 n down-convert the incoming RFsignals to generate IF or baseband signals. The IF or baseband signalsare sent to the RX processing circuitry 220, which generates processedbaseband signals by filtering, decoding, and/or digitizing the basebandor signals. The RX processing circuitry 220 transmits the processedbaseband signals to the controller/processor 225 for further processing.

The TX processing circuitry 215 receives analog or digital data (such asvoice data, web data, e-mail, or interactive video game data) from thecontroller/processor 225. The TX processing circuitry 215 encodes,multiplexes, and/or digitizes the outgoing baseband data to generateprocessed baseband or IF signals. The RF transceivers 210 a-210 nreceive the outgoing processed baseband or IF signals from the TXprocessing circuitry 215 and up-converts the baseband or IF signals toRF signals that are transmitted via the antennas 205 a-205 n.

The controller/processor 225 can include one or more processors or otherprocessing devices that control the overall operation of the eNB 102.For example, the controller/processor 225 could control the reception offorward channel signals and the transmission of reverse channel signalsby the RF transceivers 210 a-210 n, the RX processing circuitry 220, andthe TX processing circuitry 215 in accordance with well-knownprinciples. The controller/processor 225 could support additionalfunctions as well, such as more advanced wireless communicationfunctions. For instance, the controller/processor 225 could support beamforming or directional routing operations in which outgoing signals frommultiple antennas 205 a-205 n are weighted differently to effectivelysteer the outgoing signals in a desired direction. Any of a wide varietyof other functions could be supported in the eNB 102 by thecontroller/processor 225. In some embodiments, the controller/processor225 includes at least one microprocessor or microcontroller.

The controller/processor 225 is also capable of executing programs andother processes resident in the memory 230, such as a basic OS. Thecontroller/processor 225 can move data into or out of the memory 230 asrequired by an executing process.

The controller/processor 225 is also coupled to the backhaul or networkinterface 235. The backhaul or network interface 235 allows the eNB 102to communicate with other devices or systems over a backhaul connectionor over a network. The interface 235 could support communications overany suitable wired or wireless connection(s). For example, when the eNB102 is implemented as part of a cellular communication system (such asone supporting 5G, LTE, or LTE-A), the interface 235 could allow the eNB102 to communicate with other eNBs over a wired or wireless backhaulconnection. When the eNB 102 is implemented as an access point, theinterface 235 could allow the eNB 102 to communicate over a wired orwireless local area network or over a wired or wireless connection to alarger network (such as the Internet). The interface 235 includes anysuitable structure supporting communications over a wired or wirelessconnection, such as an Ethernet or RF transceiver or receiver.

The memory 230 is coupled to the controller/processor 225. Part of thememory 230 could include a RAM, and another part of the memory 230 couldinclude a Flash memory or other ROM.

Although FIG. 2 illustrates one example of eNB 102, various changes maybe made to FIG. 2. For example, the eNB 102 could include any number ofeach component shown in FIG. 2. As a particular example, an access pointcould include a number of interfaces 235, and the controller/processor225 could support routing functions to route data between differentnetwork addresses. As another particular example, while shown asincluding a single instance of TX processing circuitry 215 and a singleinstance of RX processing circuitry 220, the eNB 102 could includemultiple instances of each (such as one per RF transceiver). Also,various components in FIG. 2 could be combined, further subdivided, oromitted and additional components could be added according to particularneeds.

FIG. 3 illustrates an example UE 116 according to this disclosure. Theembodiment of the UE 116 illustrated in FIG. 3 is for illustration only,and the UEs 111-115 of FIG. 1 could have the same or similarconfiguration. However, UEs come in a wide variety of configurations,and FIG. 3 does not limit the scope of this disclosure to any particularimplementation of a UE. In one or more embodiments of this disclosure,the UE 116, or any of UEs 111-115, can be a television.

As shown in FIG. 3, the UE 116 includes an antenna 305, a radiofrequency (RF) transceiver 310, transmit (TX) processing circuitry 315,a microphone 320, and receive (RX) processing circuitry 325. The UE 116also includes a speaker 330, a main processor 340, an input/output (I/O)interface (IF) 345, a keypad 350, a display 355, and a memory 360. Thememory 360 includes a basic operating system (OS) program 361 and one ormore applications 362.

The RF transceiver 310 or receiver receives, from the antenna 305, anincoming RF signal transmitted by an eNB of the network 100. The RFtransceiver 310 or receiver down-converts the incoming RF signal togenerate an intermediate frequency (IF) or baseband signal. The IF orbaseband signal is sent to the RX processing circuitry 325, whichgenerates a processed baseband signal by filtering, decoding, and/ordigitizing the baseband or IF signal. The RX processing circuitry 325transmits the processed baseband signal to the speaker 330 (such as forvoice data) or to the main processor 340 for further processing (such asfor web browsing data).

The TX processing circuitry 315 receives analog or digital voice datafrom the microphone 320 or other outgoing baseband data (such as webdata, e-mail, or interactive video game data) from the main processor340. The TX processing circuitry 315 encodes, multiplexes, and/ordigitizes the outgoing baseband data to generate a processed baseband orIF signal. The RF transceiver 310 receives the outgoing processedbaseband or IF signal from the TX processing circuitry 315 andup-converts the baseband or signal to an RF signal that is transmittedvia the antenna 305.

The main processor 340 can include one or more processors or otherprocessing devices and execute the basic OS program 361 stored in thememory 360 in order to control the overall operation of the UE 116. Forexample, the main processor 340 could control the reception of forwardchannel signals and the transmission of reverse channel signals by theRF transceiver 310, the RX processing circuitry 325, and the TXprocessing circuitry 315 in accordance with well-known principles. Insome embodiments, the main processor 340 includes at least onemicroprocessor or microcontroller.

The main processor 340 is also capable of executing other processes andprograms resident in the memory 360. The main processor 340 can movedata into or out of the memory 360 as required by an executing process.In some embodiments, the main processor 340 is configured to execute theapplications 362 based on the OS program 361 or in response to signalsreceived from eNBs or an operator. The main processor 340 is alsocoupled to the I/O interface 345, which provides the UE 116 with theability to connect to other devices such as laptop computers andhandheld computers. The I/O interface 345 is the communication pathbetween these accessories and the main processor 340.

The main processor 340 is also coupled to the keypad 350 and the displayunit 355. The operator of the UE 116 can use the keypad 350 to enterdata into the UE 116. The display 355 may be a liquid crystal display orother display capable of rendering text and/or at least limitedgraphics, such as from web sites. In one embodiment, the keypad 350could also be a touchscreen. The touchscreen could include a touchpanel, a (digital) pen sensor, a key, or an ultrasonic input device. Thetouchscreen could recognize, for example, a touch input in at least onescheme among a capacitive scheme, a pressure sensitive scheme, aninfrared scheme, or an ultrasonic scheme. The touchscreen could alsoinclude a control circuit. In the capacitive scheme, the touchscreencould recognize touch or proximity.

The memory 360 is coupled to the main processor 340. Part of the memory360 could include a random access memory (RAM), and another part of thememory 360 could include a Flash memory or other read-only memory (ROM).

Although FIG. 3 illustrates one example of UE 116, various changes maybe made to FIG. 3. For example, various components in FIG. 3 could becombined, further subdivided, or omitted and additional components couldbe added according to particular needs. As a particular example, themain processor 340 could be divided into multiple processors, such asone or more central processing units (CPUs) and one or more graphicsprocessing units (GPUs). Also, while FIG. 3 illustrates the UE 116configured as a mobile telephone or smartphone, UEs could be configuredto operate as other types of mobile or stationary devices. In anotherexample embodiment, when UE 116 is a television, the UE 116 may notinclude a transceiver, keypad, or microphone. The UE 116 may include areceiver or decoder without a transmitter or encoder.

FIG. 4 illustrates an example system 400 for processing an image 402according to an embodiment of this disclosure. The embodiment of thesystem 400 illustrated in FIG. 4 is for illustration only, the system400 comes in a wide variety of configurations, and FIG. 4 does not limitthe scope of this disclosure to any particular implementation of asystem for processing an image.

In an embodiment of this disclosure, system 400 includes a detectionmodule 404 and a post-processing module 406. The detection module 404and post-processing module 406 can be implemented as part of or by aprocessor, such as, for example, main processor 340 as shown in FIG. 3.The image 402 can be a standard dynamic range (SDR) image. In otherembodiments, other types of image can be used, such as a high dynamicrange (HDR) image.

The detection module 404 can detect the relevant portions, which may bedesirable for enhancement, of an image 402. For example, in oneembodiment, the detection module 404 can identify only the whiteportions from image 402. These white portions and the portionssurrounding the white portions are enhanced by post-processing module404. The enhancement can be increasing the luminance (nits) of therelevant portions of the image 402.

In this example embodiment, the white portions are the relevantportions. The relevant portions can be identified by a predeterminedvalue. For example, the relevant portions can be identified by aluminance In this example, the portions of image 402 with a luminancegreater than a threshold level can be identified as relevant portions.The detection module 404 can segment such relevant portions from otherparts of the image 402. In other examples, the detection module 404 mayidentify portions with a luminance less than a threshold level. In otherembodiments, the detection module 404 may identify portions ofchrominance compared to threshold levels. The detection module 404 canuse portions of chrominance, luminance, or both when identifyingportions for enhancement. The detection module 404 can use differentmasks to identify the relevant portions of image 402.

The post-processing module 406 can enhance the relevant portions of theimage 402 by applying an enhancement factor to the relevant portions.For example, in an image where white portions are identified as relevantportions, post-processing module can enhance the white portions usingthe enhancement factor.

FIG. 5 illustrates an example image 500 to be enhanced according to anembodiment of this disclosure. The embodiment of the image 500illustrated in FIG. 5 is for illustration only, the image 500 comes in awide variety of configurations, and FIG. 5 does not limit the scope ofthis disclosure to any particular implementation of a system forprocessing an image.

In an embodiment of this disclosure, image 500 includes relevantportions 502 and non-relevant portions 504. The image 500 is an imagebefore a detection module, such as, for example, detection module 404 asshown in FIG. 4, identifies relevant portions. In one example, image 500can be an SDR image.

FIG. 6 illustrates an example mask 600 for enhancement according to anembodiment of this disclosure. The embodiment of the mask 600illustrated in FIG. 6 is for illustration only, the mask 600 comes in awide variety of configurations, and FIG. 6 does not limit the scope ofthis disclosure to any particular implementation of a system forprocessing an image.

In an embodiment of this disclosure, mask 600 includes relevant portions602 and non-relevant portions 604. The mask 600 is a mask of an imageafter a detection module, such as, for example, detection module 404 asshown in FIG. 4, identifies relevant portions. In different examples,mask 600 can be a mask of an HDR image or an SDR image.

In one example embodiment, a detection module can use YCbCr color spacefor detection of the relevant portions 602 close to white. In thisexample, the image has been converted from RGB to YCbCr. In differentembodiments, the detection module can use images in RGB and LAB colorspaces. In the YCbCr space, the detection module can identify portionsof the image (the mask) to be enhanced by using the following equations.

The masks of Y, Cb, Cr and the final overall mask from all componentscan be referred to as mask_(y), mask_(Cb), mask_(Cr), and mask_(o),respectively. Each of these masks is an array of size W×H where W is awidth of the original input image, such as image 502 as shown in FIG. 5,and H is a height of the input image. The input image can be an HDR orSDR image.

Mask in Luminance (Y):

-   mask_(y)=1 if Y>Y>YThr_(Max)-   mask_(y)=0 if Y<YThr_(Min)-   mask_(y)=(Y−YThr_(Min))/(YThr_(Max)−YThr_(Min)) otherwise

Mask in Cb:

-   mask_(Cb)=0 if Cb<=CbThr_(Min)−CbGuard-   mask_(Cb)=1−((CbThr_(Min)−Cb)/CbGuard) if CbThr_(Min)−CbGuard-   mask_(Cb)=1 if CbThr_(Min)<=Cb<=CbThr_(Max)-   mask_(Cb)=1−((Cb−CbThr_(Max))/CbGuard) if    CbThr_(Max)<=Cb<=CbThr_(Max)+CbGuard-   mask_(Cb)=0 if CbThr_(Max)+CbGuard<=Cb

Mask in Cr:

-   mask_(Cr)=0 if Cr <=CrThr_(Min)−CrGuard-   mask_(Cr)=1−((CrThr_(Min)-Cr)/CrGuard) if    CrThr_(Min)−CrGuard<Cr<CrThr_(Min)-   mask_(Cr)=1 if CrThr_(Min)<=Cr<=CrThr_(Max)-   mask_(Cr)=1−((Cr−CrThr_(Max))/CrGuard) if    CrThr_(Max)<=Cr<=CrThr_(Max)+CrGuard-   mask_(Cr)=0 if CrThr_(Max)+CrGuard<=Cr

Overall Mask:

-   m_(o)=mask_(y).*mask_(Cb).*mask_(Cr)

In the above expressions, the pixel location (i,j) has been omitted fromeach mask for notational simplicity (e.g., Y is used instead of Y(i,j)).Thr_(Max) and Thr_(Min) denote the maximum and minimum thresholds fordefining a soft mask for each mask. Guard denotes the guard band fordefining a soft mask for each mask.

FIGS. 7A and 7B illustrate example charts 700 and 702 for enhancementportions according embodiments of this disclosure. The embodiments ofthe charts 700 and 702 illustrated in FIGS. 7A and 7B are forillustration only. Charts 700 and 702 come in a wide variety ofconfigurations, and FIGS. 7A and 7B do not limit the scope of thisdisclosure to any particular implementation of a system for processingan image.

In an embodiment of this disclosure, chart 700 illustrates luminanceY,_(in) values corresponding to mask_(y) values. The chart 702illustrates luminance Cb,_(in) values corresponding to mask_(cb) values.Even though only charts for deriving the masks for Y and Cb are shown,charts for Cr may be similar to Cb. In this example, the mask in chart700 for Y is a soft and not binary mask, since a hard mask may causeartifacts at the mask boundaries in the image. In different embodiments,the detection module may also use hard masks. In many images, thevariation across edges is gradual, so a soft mask would be beneficialunder these circumstances.

For Cb (and similarly for Cr), CbThr_(Min), CbThr_(Max), CbGuard definethe thresholds and length of guard band for soft mask in Cb space. Inthis example embodiment, charts 700 and 702 sue YThr_(Max)=200;YThr_(Min)=150; CbThr_(Min)=CrThr_(Min)=118;CbThr_(Max)=CrThr_(Max)=138; and CbGuard=CrGuard=20. These valuesenhance the portion in the vicinity of white color because pure whitecolor is represented as (255,128,128) in YCbCr space. Indifferentembodiments, different values and thresholds can be used for masks by adetection module.

In this example, the mask is a gray-scale picture with real-valued pixelvalues between 0 and 1. The brighter portions in the mask (closer to 1)depict the portion to be enhanced by the post-processing enhancementmodule, while the darker portions (closer to 0) remain untouched.

FIG. 8 illustrates an example process 800 for post-processingenhancement of an image according to an embodiment of this disclosure.The embodiment of the process 800 illustrated in FIG. 8 is forillustration only, the process 800 comes in a wide variety ofconfigurations, and FIG. 8 does not limit the scope of this disclosureto any particular implementation of a system for processing an image.

In this example embodiment, process 800 illustrates enhancing Red color.The other two colors Blue and Green are enhanced in a similar manner.While the masks shown herein are for YCbCr spaces (or in other colorspaces), in some embodiments, enhancement may be performed in the {R,G,B} color space to avoid any color shifts, which may have happened ifYCbCr is used for enhancement.

At block 802, a post-processing module can calculate a boosting factor(BF) for each pixel with input luminance Y_(in) using the followingequations:

$\begin{matrix}{Y_{out} = {A + {B( \frac{Y_{in} - A}{B} )}^{\frac{1}{\alpha}}}} & (1) \\{{BF} = {Y_{out}/Y_{in}}} & (2)\end{matrix}$

The luminance pixels in the range A to 255 are boosted. B=255−A; and α(typically a>=1) is the boosting strength.

In one example embodiment, calculating a boosting factor providesenhancement to portions an image that are greater in brightness in theimage compared to other portions of the image. The portions of greaterbrightness are close to white and the enhancement allows these portionsto look whiter. One embodiment avoids calculating a BF for Cb and Cr, asthese BFs can lead to unwanted color changes in the image.

Equation 2 is just one example of an equation used for boosting(enhancing) the whiter portions with higher brightness. In differentembodiments, different equations can be designed as well. In equation 1,the parameter α can be controlled by a colorist or end-user and governsthe degree of white enhancement. In other embodiments, a can be preset.

At block 804, the input R_(in), G_(in), or B_(in) for the pixels ismultiplied by the boosting factor obtained at block 802 to obtainintermediate enhanced chromatics for each pixel. The intermediateenhanced chromatics for each pixel are shown as:

R ₁ =BF*R _(in)

G ₁ =BF*G _(in)

B ₁ =BF*B _(in)

At block 806, the intermediate enhanced chromatics R₁, G₁, or B₁ foreach pixel are multiplied by the mask m_(o) to create a masked enhancedinput. At block 808, the original pixel values R_(in), G_(in), or B_(in)are multiplied by the mask m_(o) to create a masked input. At block 810,the results of block 806 and 808 are summed as follows:

R ₂ =m _(o) *R ₁+(1−m _(o))R _(in)

G ₂ =m _(o) *G ₁+(1−m _(o))G _(in)

B ₂ =m _(o) * B+(1−m _(o))B _(in)

At block 812, the enhanced pixels R₂, G₂, or B₂ are clipped for allvalues in RGB space, which may have gone above 255 due to boosting, asfollows:

R _(out)=min(R ₂, 255)

G _(out)=min(G ₂, 255)

B _(out)=min(B ₂, 255)

In one example embodiment, the range of R, G, B is from 0-255 is usedbecause of 8-bit SDR displays. In other embodiments, other ranges can beused.

FIG. 9 illustrates an example chart 900 for input and output luminancecharacteristics according embodiments of this disclosure. The embodimentof the chart 900 illustrated in FIG. 9 is for illustration only. Chart900 comes in a wide variety of configurations, and FIG. 9 does not limitthe scope of this disclosure to any particular implementation of asystem for processing an image.

In an embodiment of this disclosure, chart 900 illustrates luminanceY_(in) input corresponding to luminance Y_(in) output. Using Equations 1and 2, FIG. 8 provides an example of a curve with A=150, and α=1.4. Inthis example, Y_(in)=200 is enhanced to approximately 212.

One or more embodiments of this disclosure provide automatic α parametertuning. The boosting strength parameter a can be chosen by a colorist oruser to decide the degree of white enhancement in the image. Setting itto a very high value (for example 10) may saturate a lot of regions inmost typical images towards white. To automate choosing a, a processormay be configured to perform the following three steps.

1) Decide a threshold t for allowing the maximum number of pixels in theimage in any channel to be saturated. In one example, pixels in theoriginal image saturated to 255 prior to boosting are ignored and notcompared to the threshold. In other examples, other sets of pixels canbe compared to the threshold.

2) With α=1 as initialization and α_(Max) as the maximum possible valueof alpha, a processor can increment α in step-size, for example, astep-size of 0.1, until the point when the number of pixels saturated isless than t in all three color spaces, and α<=α_(Max).

3) The processor is configured to output the image.

One or more embodiments of this disclosure provide mask generation inother color spaces. The embodiments described in this disclosuregenerated masks for Cb and Cr in the YCbCr color space. Alternativelythe masks can be generated based on the a and b components of the LABcolor space. For each of the a and b components, a trapezium like maskcan be designed similar to those for Cb and Cr. The center could be at“0”, and the thresholds aThrMin, aThrMax (and similarly for b) can bechosen as “5” and guard band of aGuard (and bGuard) to be “10”'. Thesevalues are just for illustration, and other values can be used.

FIG. 10 illustrates an example system 1000 using an HDR image 1002 forprocessing an SDR image 1004 according to an embodiment of thisdisclosure. The embodiment of the system 1000 illustrated in FIG. 10 isfor illustration only, the system 1000 comes in a wide variety ofconfigurations, and FIG. 10 does not limit the scope of this disclosureto any particular implementation of a system for processing an image.

One or more embodiments of this disclosure utilize HDR information ofHDR image 1002 for detecting relevant portions of an image and usingthose portions for enhancement of an SDR image. In this example, the SDRimage 1004 is created by tone mapping 1006 the HDR image 1002. When theHDR image 1002 is also available in addition to the SDR image 1004, thedetection of the white, or near-white region will be more accurate inthe HDR space as compared to the SDR space. Due to tone mapping(conversion from HDR to SDR), there is some information loss for all thepixels. One or more embodiments provide a detection mask in the HDRimage space, and then use the mask is used to perform the enhancement inthe SDR image.

In an embodiment of this disclosure, system 1000 includes a detectionmodule 1008 and a post-processing module 1010. The detection module 1008and post-processing module 1010 can be implemented as part of or by aprocessor, such as, for example, main processor 340 as shown in FIG. 3.The detection module 1008 identifies relevant portions of the HDR image1002 for enhancement to form metadata with information about therelevant portions of the HDR image. The post-processing module uses themetadata provided by the detection module 1008 regarding the relevantportions of the HDR image 1002 to enhance those relevant portions on theSDR version of the image 1004.

In the HDR image 1002, at higher luminance values (e.g., at 4000 nits),a human eye contrast sensitivity function (CSF) will be able todistinguish fewer details in contrast. When using a linear tone mapping,or a tone mapping function not perceptually designed is used to tone mapthe HDR image 1002 to SDR image 1004, and contrast sensitivity of theeye will be more (e.g., at around 200 nits) after tone mapping in theSDR image 1004. During enhancement (boosting), that contrast level inthe white area of the enhanced SDR image can be made reduced as comparedto the tone mapped SDR image 1004, since in the original HDR image 1002,the contrast was less visible to the human eyes.

The metadata from the HDR image 1002 can be used to find the minimumY_(min,out) luminance value for the enhanced SDR image that shouldprovide the same CSF as in HDR image 1002. In one example, the maxluminance value can go to 255 (in 8-bit RGB images). The boostingparameter a can be computed as:

$\begin{matrix}{{Y_{\min,{out}} = {A + {B( \frac{Y_{\min,{{in} - A}}}{B} )}^{\frac{1}{\alpha}}}},} & \;\end{matrix}$

where Y_(min,out) is obtained using the CSF function.

None of the description in this application should be read as implyingthat any particular element, step, or function is an essential elementthat must be included in the claim scope. The scope of patented subjectmatter is defined only by the claims. Moreover, none of the claims isintended to invoke 35 U.S.C. §112(f) unless the exact words “means for”are followed by a participle.

What is claimed is:
 1. A user equipment (UE), comprising: a receiverconfigured to receive a standard dynamic range (SDR) image and metadatarelated to an HDR image; a processor configured to: identify relevantportions of the SDR image to be enhanced based on the metadata relatedto the HDR image; increase an intensity of the relevant portions of theSDR image to create an enhanced SDR image; and output the enhanced SDRimage to a display.
 2. The UE of claim 1, wherein performing tonemapping on the HDR image creates the SDR image.
 3. The UE of claim 1,wherein the intensity is increased for one or more of a luminance orchrominance of the SDR image.
 4. The UE of claim 1, wherein the metadataincludes one or more of a luminance mask or chrominance mask of the HDRimage.
 5. The UE of claim 1, wherein the processor configured toincrease the intensity of the relevant portions of the SDR image tocreate the enhanced SDR image comprises the processor configured to:determine a boosting factor using a luminance component for each pixelof the relevant portions of the HDR image.
 6. The UE of claim 4, whereinthe processor configured to increase the intensity of the relevantportions of the SDR image to create the enhanced SDR image comprises theprocessor configured to: apply the boosting factor to one or more colorchromatics to form intermediate color chromatics.
 7. The UE of claim 5,wherein the processor configured to increase the intensity of therelevant portions of the SDR image to create the enhanced SDR imagecomprises the processor configured to: apply the one or more of aluminance or chrominance mask to the one or more color chromatics tocreate a masked input; apply an inverse of one or more of a luminance orchrominance mask to the intermediate color chromatics to create a maskedenhanced color input; and sum the masked color input and the maskedenhanced input to create the enhanced color chromatics.
 8. The UE ofclaim 7, wherein the processor configured to increase the intensity ofthe relevant portions of the SDR image to create the enhanced SDR imagecomprises the processor configured to: reduce any of the enhanced colorchromatics above a predetermined value to the predetermined value. 9.The UE of claim 4, wherein the processor configured to determine theboosting factor using the luminance component for each pixel of therelevant portions of the HDR image comprises the processor configuredto: determine the boosting factor where a number of saturated pixels ofthe SDR image are less than a threshold number of pixels.
 10. The UE ofclaim 7, wherein the processor configured to increase the intensity ofthe relevant portions of the SDR image to create the enhanced SDR imagecomprises the processor configured to: identify any saturated pixels ofthe HDR image; and saturate pixels of the SDR image corresponding to thesaturated pixels of the HDR image.
 11. A method for image enhancement,comprising: receiving a standard dynamic range (SDR) image and metadatarelated to an HDR image; identifying relevant portions of the SDR imageto be enhanced based on the metadata related to the HDR image;increasing an intensity of the relevant portions of the SDR image tocreate an enhanced SDR image; and outputting the enhanced SDR image to adisplay.
 12. The method of claim 11, wherein performing tone mapping onthe HDR image creates the SDR image.
 13. The method of claim 11, whereinthe intensity is increased for one or more of a luminance or chrominanceof the SDR image.
 14. The method of claim 11, wherein the metadataincludes one or more of a luminance mask or chrominance mask of the HDRimage.
 15. The method of claim 11, wherein increasing the intensity ofthe relevant portions of the SDR image to create the enhanced SDR imagecomprises: determining a boosting factor using a luminance component foreach pixel of the relevant portions of the HDR image.
 16. The method ofclaim 14, wherein increasing the intensity of the relevant portions ofthe SDR image to create the enhanced SDR image comprises: applying theboosting factor to one or more color chromatics to form intermediatecolor chromatics.
 17. The method of claim 15, wherein increasing theintensity of the relevant portions of the SDR image to create theenhanced SDR image comprises: applying the one or more of a luminance orchrominance mask to the one or more color chromatics to create a maskedinput; applying an inverse of one or more of a luminance or chrominancemask to the intermediate color chromatics to create a masked enhancedcolor input; and summing the masked color input and the masked enhancedinput to create the enhanced color chromatics.
 18. The method of claim17, wherein increasing the intensity of the relevant portions of the SDRimage to create the enhanced SDR image comprises: reducing any of theenhanced color chromatics above a predetermined value to thepredetermined value.
 19. The method of claim 14, wherein determining theboosting factor using the luminance component for each pixel of therelevant portions of the HDR image comprises: determining the boostingfactor where a number of saturated pixels of the SDR image are less thana threshold number of pixels.
 20. The method of claim 17, whereinincreasing the intensity of the relevant portions of the SDR image tocreate the enhanced SDR image comprises: identifying any saturatedpixels of the HDR image; and saturating pixels of the SDR imagecorresponding to the saturated pixels of the HDR image.